Skip to content

Add AgentX cumulative convergence diagnostics / 新增 AgentX 累计收敛诊断#2296

Draft
cquil11 wants to merge 2 commits into
mainfrom
codex/agentx-uncertainty-runner-mvp
Draft

Add AgentX cumulative convergence diagnostics / 新增 AgentX 累计收敛诊断#2296
cquil11 wants to merge 2 commits into
mainfrom
codex/agentx-uncertainty-runner-mvp

Conversation

@cquil11

@cquil11 cquil11 commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Preserve the existing non-overlapping 10-minute window and root-trajectory coverage diagnostics.
  • Add 5-minute cumulative checkpoints for P75/P90 TTFT, end-to-end latency, and interactivity.
  • Define stabilization as the earliest checkpoint with at least 20 minutes of later confirmation where every later cumulative estimate stays inside a symmetric multiplicative ±5% band around the final complete checkpoint.
  • Emit the stabilization time, metric-specific valid-request count, post-stabilization minimum/maximum, and maximum later deviation.
  • Keep the nested schema explicit under request_metrics.stability.convergence and add boundary, reciprocal-symmetry, insufficient-duration, late-excursion, horizon, and sample-count tests.

Statistical interpretation

This is a retrospective diagnostic of convergence within one run. It is not a confidence interval, a rerun prediction interval, or a substitute for independent randomized repetitions. The displayed span is the range of cumulative estimates after stabilization, not the total range of individual requests or fixed windows.

The band is evaluated with absolute log ratios, so latency and its reciprocal interactivity stabilize at the same checkpoint.

Companion rollout

Validation

中文说明

  • 保留现有的 10 分钟互不重叠时间窗口诊断和根轨迹覆盖诊断。
  • 针对 P75/P90 TTFT、端到端延迟和交互性指标,每 5 分钟计算一次累计指标。
  • 将“稳定”定义为:某个最早检查点之后至少还有 20 分钟的确认期,且后续所有累计估计值都位于最终完整检查点估计值的对称乘法 ±5% 范围内。
  • 输出达到稳定的时间、该指标的有效请求数、稳定后的最小值/最大值,以及后续最大偏差。
  • request_metrics.stability.convergence 下保留清晰的嵌套结构,并补充边界、倒数对称性、时长不足、后期越界、完整时间范围及样本计数测试。

统计解释

该指标是对单次运行内部收敛情况的回溯性诊断,并非置信区间、复跑预测区间,也不能替代独立随机重复实验。图中范围表示达到稳定后各累计估计值的跨度,而不是单个请求或固定时间窗口的完整取值范围。

稳定范围通过绝对对数比计算,因此延迟及其倒数形式的交互性指标会在同一个检查点达到稳定。

配套发布

验证

Compute retrospective cumulative-prefix stabilization for P75/P90 TTFT, end-to-end latency, and interactivity while preserving the existing fixed-window diagnostics.

中文:为 P75/P90 TTFT、端到端延迟和交互性指标新增回溯性累计前缀收敛诊断,并保留现有固定时间窗口诊断。
@cquil11 cquil11 changed the title Add AgentX fixed-window stability diagnostics Add AgentX cumulative convergence diagnostics / 新增 AgentX 累计收敛诊断 Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant